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Abstract 

Photometric  signatures  of  satellites  collected  from  small  aperture  optical  systems  have  the 
potential  of  providing  valuable  satellite  information  including  operational  status  and  attitude.  Low 
cost,  small  aperture  systems  provide  the  capability  to  be  transportable  and  therefore  used  in  a  wide 
variety  of  deployment  scenarios.  This  research  describes  the  benefits  of  prior  knowledge  of  the 
satellite  under  consideration  including  general  size,  shape  and  typical  material  properties.  By 
constraining  the  problem  to  a  discrete  set  of  satellite  attitudes,  curve  matching  algorithms  can  then 
be  used  to  compare  the  measured  photometric  signature  against  those  in  the  forward  modeled 
database.  TASAT  simulations  were  used  to  create  a  database  of  forward  modeled  photometric 
signatures.  In  this  paper,  we  explore  two  curve  matching  techniques.  The  first  method  was 
Kendall's  Tau  which  compares  the  shapes  of  two  curves.  The  second  method  was  the  Spectral 
Angle  Mapper  (SAM)  which  measures  the  similarity  of  two  curves  that  have  been  processed  to 
remove  their  mean  values.  The  key  findings  of  this  study  are  the  generation  of  a  database  of 
signatures  to  which  a  measured  signal  can  be  compared  to  extract  attitude  information  about  the 
satellite  during  a  specific  pass. 


1.  Introduction 

Photometry  is  the  measurement  of  an  object’s  apparent  brightness,  usually  measured  within  a  specific  calibrated 
wavelength  band.  A  photometric  signature  is  produced  by  several  measurements  of  the  object’s  brightness  over  a 
known  time  period.  For  this  study,  the  time  period  is  the  duration  of  the  satellite  pass  from  horizon  to  horizon.  For 
geometric  considerations,  when  the  satellite  attitude  is  in  a  ground-pointing  orientation  above  an  area  of  interest,  the 
satellite’s  surface  orientations  with  respect  to  the  optical  sensor  and  the  source  of  illumination  (e.g.  the  sun)  varies 
creating  increasing  or  decreasing  intensities,  and  thus  produces  a  photometric  signature  that  varies  as  a  function  of 
the  attitude  of  the  satellite.  The  objective  of  this  feasibility  study  was  to  determine  if  the  attitude  of  a  particular 
satellite  produces  a  unique  photometric  signature  that  could  be  matched  to  a  database  of  previously  generated 
satellite  signature  models  to  within  2°  of  a  satellite  attitude  change  and  a  90%  classification  accuracy. 

The  findings  of  this  paper  provide  insight  into  the  shape  and  attitude  recovery  problem  in  that  there  are  many 
sources  of  variation  in  the  signatures.  The  first  problem  being  that  space  materials  are  not  always  accurate  on  the 
models  used  for  the  simulation  of  the  database  of  signatures.  This  is  important  when  trying  to  simulate  satellites  that 
are  not  owned  or  operated  by  cooperative  entities.  The  second  source  of  signal  degradation  is  the  calibration 
accuracy  of  the  atmospheric  transmission.  This  value  is  measured  on  the  sensor  by  the  use  of  calibration  stars,  and  is 
desired  to  be  less  than  5%  for  good  radiometry,  and  1%  for  excellent  radiometry.  Lastly,  every  possible  position  of 
the  satellite  cannot  be  simulated  in  a  timely  fashion,  and  thus  a  good  portion  of  the  database  should  reflect 
orientations  that  are  of  value  to  the  observing  location  for  signature  comparison.  This  provides  a  limiting  number  of 
solutions  and  highly  constrains  the  problem  down  to  a  specific  number  of  signatures.  Results  show  that  with 
moderate  calibration  errors,  of  +  20%,  the  Spectral  Angle  Mapper  (SAM)  technique  matches  correctly  to  the 
database  with  a  90%  success  rate  for  a  cylindrical  type  of  satellite  in  a  high-elevation  pass. 
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Public  reporting  burden  for  the  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information, 
including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington 
VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  a  penalty  for  failing  to  comply  with  a  collection  of  information  if  it 
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The  remainder  of  this  paper  is  organized  as  follows.  Section  2  describes  the  end-to-end  modeling  and  simulation 
setup,  Section  3  deals  with  the  signature  generation  process  with  TASAT  and  Raven  class  telescopes,  Section  4 
discusses  the  curve  matching  results  to  the  database  and  Section  5  discusses  the  conclusions. 

2.  End-to-End  Modeling  and  Simulation 

Fig.  1  illustrates  the  end-to-end  modeling  and  simulation  environment  used  to  test  and  evaluate  the  two  curve 
matching  algorithms.  AFRIT  s  j'Track[l]  software  tool  computes  a  set  of  possible  satellite  orientations  in  6  Degrees 
of  Freedom  (6DoF),  commensurate  satellite  attitude  geometries  or  orientation,  and  computed  Earth  tracks.  The 
attitude  geometries  and  Earth  tracks  are  stored  in  two  separate  caches.  The  Earth  Track  Cache  is  maintained  until 
the  signature  matching  process  has  completed.  The  information  stored  in  the  satellite  Geometry  Cache  is  used  as 
input  into  the  Time-domain  Analysis  and  Simulation  for  Advanced  Tracking  (TASAT)[2]  software,  which  then 
produces  photometric  signatures  for  each  of  the  potential  satellite  orientations,  from  the  Geometry  Cache ,  that  are 
stored  in  the  Photometric  Signature  Cache. 

For  this  small  aperture  study,  a  Raven  class  telescope  was  modeled  as  a  Celestron  11”,  Apogee  Alta  U47  (1024  x 
1024,  13pm  pixels)  stationed  at  the  Remote  Maui  Experiment  (RME)  located  in  Kihei,  Maui.  In  the  simulation,  a 
Raven  class  sensor  generates  a  raw  scene-simulated  satellite  photometric  curve.  The  Photometric  Signature 
Atmospheric  Effect  Compensator  algorithm  receives  and  compensates  the  signature  for  atmospheric  absorption  and 
scattering  effects.  For  the  base  model,  the  only  source  of  noise  is  Additive  White  Gaussian  Noise  (AWGN)  that 
degrades  the  signature  as  a  read  noise  term.  The  Photometric  Signature  Comparator  algorithm  then  compares  the 
calibrated  photometric  signature  to  the  Photometric  Signature  Cache  and  returns  the  best  match  within  the  2°  angle- 
estimation  accuracy  requirement.  It  should  be  noted  that  observations  of  calibration  stars  in  conjunction  with  each 
pass  is  necessary  to  properly  calibrate  the  measured  photometric  signatures.  The  atmospheric  degradation  is 
removed  by  division  of  the  transmission  curve  from  the  measured  signature  to  produce  an  exo- atmospheric  intensity. 
This  exo-atmospheric  intensity  signature  is  then  compared  to  the  simulated  signatures  via  the  Photometric  Signature 
Comparator.  The  results  from  the  signature  comparator  feed  directly  into  the  ground  track  solutions  provided  in  the 
Earth  Track  Cache  showing  near  real-time  orientation  results. 
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Figure  1.  Processing  Flow  Simulation  for  Determining  Earth  Tracks 


Two  distinct  3-D  satellite  bus  type  models  were  built  in  the  New  Solid  Model  (NSM)  format  with  the  Solid 
Modeling  Tool  (SMT)  to  test  the  photometric  signature  matching  algorithms.  The  models  consist  of  a  body-pointing 
cylindrical  satellite  system  and  a  body-pointing  cubic  satellite  system.  These  features  are  representative  of  many 
earth-observing  satellites.  Fig.  2,  shows  a  cylindrical  satellite  bus  representing  a  telescope  imaging  system,  which 


provides  a  unique  signature  for  specific  phase  angles.  Fig.  3,  shows  a  cubic  satellite  bus  representing  a  dual  imaging 
bus  with  protruding  shade  cones  or  antennas.  The  cubic  satellite  bus  presents  a  more  difficult  photometric  signature 
matching  problem  than  the  cylindrical  imager  due  to  the  symmetry  of  the  satellite. 


Figure  2.  Cylinder  Satellite  Model  Figure  3.  Cube  Satellite  Model 


First,  a  satellite  orbit  is  generated  simulating  an  earth-observing  pass  for  a  particular  area  of  interest  on  the  ground. 
Using  the  models  in  Fig.  2  and  3,  built  with  the  +Z-axis  in  the  payload  direction,  the  jSim  software  tool  computes 
the  satellite  orbit  based  on  two-line  element  sets  (TLE).  After  the  orbit  is  computed,  the  location  of  the  specific  area 
of  interest  is  used  to  generate  the  satellite  body-point  angles  required  to  image  that  particular  area.  This  rotation 
value,  estimated  as  a  cross-track  roll  angle,  is  used  to  generate  the  Geometry  Cache  required  to  populate  the 
Photometric  Signature  Cache.  The  +Z-axis  in  the  body  frame  of  the  satellite  is  rotated  away  from  the  nadir  vector  to 
generate  the  roll  values,  with  one  or  more  of  these  rotations  pointing  at  the  ground  observer.  The  cross-track  values 
are  a  simplification  to  the  general  problem  of  using  in-track  (pitch)  and  cross- track  (roll),  decreasing  the  amount  of 
angles  required  for  simulation.  The  remainder  of  this  report  will  focus  on  the  results  from  the  Cylinder  Satellite 
model  only. 


Solid  Modeling  Tool  (SMT)  Reflectance  Curves 

The  materials  that  were  selected  from  the  SMT  material  database  in  developing  the  satellite  models  were 
Aluminized  Kapton,  Multi-layer  Insulation  (MLI),  Solar  Stack  and  Blue  Silicon  Solar  Cells.  The  material 
reflectance  curves  for  the  Cylinder  and  Cube  Satellites  are  displayed  in  the  figures  below.  The  reflectance  values  are 
provided  by  the  Satellite  Assessment  Center  (SatAC)  in  a  file  distributed  with  both  TASAT  and  SMT  called  the 
Materials  Database.  Fig.  4  shows  the  reflectance  of  Aluminized  Kapton  that  is  used  as  the  outer  material  for  the 
main  body  of  Cylinder  Sat.  Fig.  5  displays  the  reflectance  curve  for  the  solar  stack  material  of  the  Cylinder  Sat.  Fig. 
6  displays  the  reflectance  curve  for  the  MLI  that  covers  the  outer  layer  of  Cube  Sat.  Lastly,  Fig.  7  shows  the  solar 
panel  reflectance  of  Cube  Sat  that  consisted  of  Blue  Silicon  Solar  Cells.  It  should  be  noted  that  the  wavelengths  of 
the  simulations  for  this  project  were  conducted  in  the  700  nm  -  1200  nm,  or  I-Band,  range. 


Figure  4.  Reflectance  function  of  Cylinder  Sat  surface  Figure  5.  Reflectance  function  of  Cylinder  Sat  solar 
material,  2  mm  Aluminized  Kapton.  panel  material,  Solar  Stack  sun  side. 


Figure  6.  Reflectance  function  of  Cube  Sat  surface  Figure  7.  Reflectance  function  of  Cube  Sat  solar  panel 
material,  Multi-layer  Insulation  (MLI)  material,  Blue  Silicon  Solar  Cell. 

It  should  be  noted  that  the  wavelengths  of  the  simulations  for  this  project  were  conducted  in  the  780  nm  -  1200  nm, 
or  I-Band,  range.  When  combined  with  the  Silicon  response  curves  of  the  camera  system  the  overall  waveband 
response  becomes  780  nm  -  980  nm.  Total  reflectance  curves  of  the  entire  system  are  not  possible  to  generate  due  to 
the  complex  relationship  between  the  components.  To  overcome  this  complexity,  TASAT  handles  the  individual  ray 
tracing  for  each  surface  and  each  pixel  with  the  ability  to  add  multiple  reflections  when  generating  the  photometric 
light  curves. 


3.  Raven-class  Photometric  Signature  Simulated  with  TASAT 

AFRIT  s  Time-domain  Analysis  and  Simulation  (TASAT)  software  was  used  to  simulate  the  calibrated  photometric 
signatures  which  would  result  from  data  collected  from  a  Raven-class  telescope.  TASAT  builds  a  detailed  optical 
tracking  scenario  given  multiple  optical  and  satellite  inputs,  and  determines  the  position  and  orientation  of  a  satellite 
in  the  Earth  Centered  Inertial  (ECI)  reference  frame.  Based  on  the  inputs,  a  ray-tracing  algorithm  within  TASAT 
computes  the  solar  reflectance  of  the  satellite  from  a  light  source  (Sun)  to  the  Raven  optical  sensor.  There  were 
constraints  placed  on  the  orbit  of  the  satellite  and  the  observing  location.  All  passes  culminated  over  the  observing 
location  at  an  elevation  greater  than  45°  and  were  in  a  terminator  period  to  eliminate  sky  foreground  complications 

Atmospheric  Errors 

Based  on  the  assumption  that  the  aperture  photometry  technique  accounts  for  all  sensor  calibration  characteristics 
such  as  removing  read  noise  and  dark  currents,  the  atmosphere  becomes  the  largest  source  of  errors.  To  simulate  this 
source  of  errors,  the  photometric  signatures  that  are  generated  are  perturbed  by  a  nominal  atmospheric  extinction  of 
0.17  and  then  corrected  with  a  measured  extinction  value  between  0.10  and  0.24,  or  alternatively  a  maximum  of 
40%  error  in  atmospheric  calibration.  The  values  that  are  less  than  the  nominal  value  of  0.17  are  underestimating  the 
atmospheric  extinction  and  thus  are  not  correcting  the  measured  signatures  enough,  while  the  values  greater  than 
0.17  are  overestimating  the  extinction  of  the  atmosphere  and  generate  larger  values  than  what  should  be  measured. 
The  difference  in  the  signature  when  multiplied  by  the  nominal  transmission  curve  and  then  divided  by  the  measured 
transmission  curve  are  the  definition  of  the  atmospheric  calibration  errors.  This  horseshoe  shape  of  the  transmission 
curve  versus  elevation  of  the  satellite  provides  larger  errors  at  the  start  and  end  of  a  pass  relative  to  the  nominal 
signal  strength  while  causing  a  scalar  offset  in  the  middle  parts  of  the  pass. 

Photometric  Signature  Matching 

Photometric  signature  matching  algorithms  were  applied  to  the  measured  photometric  signature  to  match  to  the 
database  of  forward  modeled  photometric  signatures  in  the  Photometric  Signature  Cache.  The  two  photometric 
signature  matching  algorithms  that  were  selected  for  this  study  are  named  TAU,  for  an  implementation  of  Kendall’s 
Tau[3],  and  the  Spectral  Angle  Mapper[4],  or  SAM.  The  TAU  algorithm  uses  Kendall’s  Tau  as  a  measure  of  the 
likeness  of  two  shapes  using  a  two-dimensional  difference  surface  and  a  correlation  type  metric.  The  SAM  method 
is  the  spectral  angle  mapper,  and  uses  an  A-dimensional  dot  product,  and  relies  on  the  mathematics  of  the  dot 
product  having  zero  angle  difference  for  two  parallel  vectors. 


Kendall’s  Tau  is  a  measure  of  the  similarity  between  two  different  shapes,  or  in  this  case,  two  different  photometric 
signatures.  This  method  requires  that  we  form  what  is  known  as  a  derivative  surface  by  piecewise  comparing  of  each 
simulated  photometric  signature  with  itself.  A  similar  surface  is  then  created  using  the  measured  signal.  Once  the  2D 
surfaces  are  retrieved  from  the  ID  signals,  we  compare  the  number  of  points  in  the  two  surfaces  that  are  the  same 
and  different.  The  photometric  signature  that  maximizes  Kendall’s  Tau,  as  this  is  value  is  a  correlation  metric,  is  the 
best  estimate.  The  valid  range  of  values  for  the  Kendall’s  Tau  are  [-1,1]  with  1  being  an  exact  match  and  -1  being  an 
exact  mirror-mismatch.  Because  the  method  relies  on  comparisons  of  slopes,  the  similarity  of  shapes  of  photometric 
signatures  is  more  important  than  absolute  values. 


Figure.  8.  Curves  used  to  demonstrate  Kendall’s  Tau  method  of  curve  matching  in  which  G(x)  is 
almost  a  monotone  transformation  of  F(x). 

Consider  the  two  sample  photometric  signatures  in  Fig.  8.  For  example,  photometric  signature  F(x)  represents  the 
measured  photometric  signature  from  an  optical  sensor,  while  photometric  signature  G(x)  represents  one  of  the 
photometric  signatures  from  our  forward-modeled  Photometric  Signature  Cache  database.  In  Fig.  8,  G(x’)-G(x)  has 
the  same  sign  as  F(x’)-F(x)  for  almost  all  pairs  (x,  x’).  While  there  are  certainly  differences,  they  do  have  roughly 
similar  shapes.  We  now  discuss  criteria  by  which  formal  definitions  of  similarity  are  established. 


F(A)  -  F(B)  >  0  I  =+l 

F(A)  -  F(B)  <  0  ■  =  -1 

F(A)  -  F(B)  =  0  =>  ■  =  0 


Figure.  9.  Construction  of  the  two-dimensional  difference  function. 

In  the  Kendall’s  Tau  method,  we  examine  how  differences  in  pairs  of  points  on  the  photometric  signatures  compare. 
Fig.  9  shows  the  results  of  comparing  a  function  F  at  two  points  A  and  B  by  subtraction  as  F(A)-F(B).  When  A=B, 
we  are  comparing  the  function  at  the  same  point,  so  the  value  is  zero,  and  thus  the  diagonal  along  all  surfaces  will  be 
zero.  The  Kendall’s  Tau  method  depends  only  on  the  fact  that  there  may  be  a  difference  in  the  values  at  the  two 
points,  and  not  on  the  magnitude  of  the  difference.  Therefore,  if  F(A)>F(B)  we  assign  the  value  +1  at  the  pair  (A,B). 
If  F(A)<F(B)  then  we  assign  -1.  This  results  in  a  two-dimensional  map  as  shown  in  Fig.  9  where  green  squares 
indicate  a  positive  difference,  red  squares  indicate  a  negative  difference  and  black  squares  represent  a  zero  location. 

Fig.  10  shows  that  the  difference  map  for  G(x)  is  nearly  the  same  as  for  F(x),  displaying  many  similar  traits 
functionally.  To  determine  how  similar  the  two  curves  are,  we  need  to  count  the  number  of  similar  and  dissimilar 
points  exist.  Thus,  where  both  maps  have  the  same  sign,  we  say  that  there  is  a  concordance,  and  denote  that  count  by 


C.  Where  the  two  maps  have  different  signs,  there  is  a  discordance  and  that  count  is  denoted  by  D.  Therefore,  we 
have  just  counted  the  number  of  matching  and  mismatching  points  in  the  two  surfaces  of  F(x)  and  G(x)  into  the 
values  of  C  and  D.  The  difference  C-D  is  then  a  measure  of  how  close  the  two  maps  are  to  each  other.  We  can 
normalize  that  quantity  by  dividing  by  the  number  of  pairs,  which  can  be  formed  from  our  N  samples. 


In  Fig.  10,  G(A)-G(B)  has  the  same  sign  as  F(A)-F(B)  for  almost  all  pairs  A,B.  This  is  an  indication  that  the  two 
photometric  signatures  are  very  similar.  To  determine  how  similar,  the  Kendall’s  Tau  value  is  computed  by  using 
the  definitions  of  C  (matching  pairs),  D  (mismatched  pairs)  and  n  (total  number  of  pair)  in  (1). 


C-D 


(i) 


Fig.  10.  The  second  signal  is  almost  a  monotone  transformation  of  the  first. 

The  second  algorithm  is  the  Spectral  Angle  Mapper  (SAM).  The  origins  of  the  SAM  metric  in  [5]  are  derived  from 
remote  sensing  data  used  to  compare  regions  of  space  based  images  taken  of  the  ground.  The  SAM  metric,  in  (1) 
becomes  an  angle  in  an  TV-dimensional  space  between  two  A-dimensional  vectors.  The  two  vectors  are  the  incoming 
data  stream  of  the  measured  signature  and  one  of  the  simulated  data,  or  database  signature,  streams.  Formally, 
denoting  the  vectors  as  Im(tj,t2,t3,  ...tN)  for  the  measured  data  and  Is(tj,t2,t3,  ...tN)  for  the  simulated  data  set,  we  form 
the  dot  product  of  the  two  vectors,  as  shown  in  (2). 

im  °  4  =  fcjnu  cos  0  (2) 

Then,  based  on  the  values  in  (2),  6  is  the  spectral  angle.  The  smaller  the  value  of  6 ,  the  more  alike  the  two  vectors, 
however,  the  specific  constraint  on  the  timing  of  the  sequences  apply.  If  the  vectors  are  not  put  on  the  same  time 
scales,  and  synchronized,  the  results  may  be  skewed.  For  this  project,  all  photometric  signatures  are  perfectly 
aligned  in  time,  and  constitute  a  continuous  data  set.  In  other  words,  there  are  no  breaks  in  the  data  stream  from 
clouds,  equipment  malfunctions,  or  operator  errors. 

Generation  of  Earth  Track/Photometric  Signature  Cache 

MATLAB  was  used  as  an  interface  to  the  j'Sim  libraries,  including  orbit  propagation,  Earth  Track  determination,  and 
satellite  orientation  methods.  The  inputs  to  j'Sim  include  the  satellite  TLE,  time  of  interest,  an  asset  location 
(latitude/longitude/altitude),  and  Raven  sensor  location.  j'Sim  uses  the  standard  Air  Force  SGP4  propagator  to 
compute  the  position  of  the  satellite  during  the  time  of  interest  to  compute  the  photometric  signature  collection 
opportunities  of  the  satellite. 


The  combined  software  tool  calculates  the  satellite  orientation  required  to  image  the  asset  location.  The  software 
then  computes  the  orientation  required  to  image  the  asset  with  only  a  roll  maneuver.  This  keeps  the  number  of 
angles  for  simulation  to  a  minimum  and  matches  the  operational  capabilities  of  many  Earth  Imaging  satellites.  The 
cross-track  roll  angle  is  found  at  the  nearest  lA  second  around  culmination.  This  distinct  orientation  angle,  or  threat 
angle,  is  then  used  as  the  basis  for  signature  generation  to  determine  if  an  asset  or  location  has  been  imaged. 

To  minimize  the  simulation  processing  time  for  this  study,  a  smaller  set  of  different  rotation  angles  are  computed 
that  encompass  the  threat  angle  and  include  the  unique  nadir  down  orientation.  The  simulation  data  set  includes  the 
rotation  angles  spanning  the  threat  angle  in  ten  2°  steps  for  a  total  of  21  unique  rotation  angles,  or  +/-  18°  of 
coverage  around  the  threat  angle.  The  nadir  down  orientation  is  included  specifically  as  a  common  satellite  cruise 
mode  and  provides  a  signature  that  could  indicate  a  satellite  is  currently  not  performing  an  imaging  mission. 

4.  Results 

Photometric  Signature  Matching  Performance 

The  simulations  performed  for  this  performance  analysis  consisted  of  two  different  satellite  bus  structures;  two 
different  elevation  passes  with  two  dramatically  different  lighting  conditions,  or  phase  angles.  In  addition,  over  21 
different  levels  of  atmospheric  calibration  errors  were  applied  to  degrade  the  signatures  before  comparison  to 
provide  a  number  of  samples  for  comparison.  The  performance  of  the  algorithms  degrades  as  the  amount  of 
calibration  error  increases,  and  can  be  seen  in  the  following  figures.  Providing  each  of  the  possible  signatures  as 
truth  for  each  calibration  error  value  produced  these  figures.  Counting  the  number  of  times  each  orientation  was 
correctly  estimated  and  when  each  pose  was  incorrectly  estimated  generates  the  success  rates.  This  process  was  done 
again  by  adding  the  equivalent  of  read-noise  in  the  form  of  zero-mean  data  Gaussian  data.  This  small  amount  of 
noise  on  the  large  values  of  the  signatures  produces  a  different  result  in  the  comparison  algorithms  based  on  the  type 
of  satellite  being  observed. 

The  performance  of  the  signature  matching  algorithms  for  a  high  elevation  pass  for  the  Cylinder  Sat  model  is  shown 
in  Fig.l  1  and  Fig.  12.  The  data  in  Fig.  11  shows  the  performance  for  an  exact  match  of  the  signatures,  while  Fig.  12 
contains  the  performance  for  a  ±2°  approximation  window  around  the  exact  match.  The  red  curve  depicts  the 
Spectral  Angle  Mapper  (SAM),  the  blue  curve  depicts  the  Kendall’s  Tau  method,  and  the  green  curve  is  an  RMS 
method  used  to  show  a  baseline  comparison.  The  results  show  that  SAM  is  the  best  matching  algorithm  for  the 
Cylinder  Sat  for  both  the  precise  orientations  and  approximate  orientations.  But,  as  the  calibration  error  increases 
from  either  over-estimation  or  under-estimation,  of  the  atmospheric  transmission,  all  the  comparison  algorithms  fall 
off  in  the  number  of  correct  satellite  orientation  estimations,  with  only  the  photometric  signatures  with  strong 
features  being  correctly  estimated.  The  strong  features  that  occur  in  the  Cylinder  Sat  case  are  glints  off  the  imaging 
tube  of  the  satellite.  Also  shown  in  the  figures  are  the  curves  that  do  not  have  read-out  noise  degradation  and  those 
that  had  a  read-out  component  comprised  of  zero-mean  Gaussian  noise  a  standard  deviation  of  25  W/sr.  The 
performance  of  the  noise  degraded  curves  is  shown  in  dashed  lines  while  the  noiseless  curves  are  solid. 
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Figure  11.  Cylinder  Sat,  High  Elevation,  Photometric  Figure  12.  Cylinder  Sat,  High  Elevation,  Photometric 

Signature  matching  Performance  Curves  (exact)  Signature  matching  Performance  Curves  (approximate) 


When  the  pass  geometry  is  moved  from  a  ‘high  elevation’  to  a  ‘low  elevation’  situation  the  results  of  the  curve 
matching  changes.  The  results  shown  in  Fig.  13  provide  the  exact  solution  match  for  the  Cylindrical  Satellite  that 
culminates  at  an  elevation  of  53°  from  the  observing  location.  This  is  in  contrast  to  the  84°  elevation  of  the  high- 
elevation  situation  previously  described.  The  exact  solution  performance  curves  show  good  performance  out  to 
±40%  calibration  errors.  The  performance  shows  that  Kendall’s  Tau  is  now  the  better  metric  when  compared  to 
SAM  for  this  instance.  The  SAM  performance  drops  more  rapidly  on  the  positive  side  of  the  calibration  errors, 
while  Kendall’s  Tau  is  nearly  always  above  90%  successful.  On  the  negative  side,  or  underestimated  errors,  SAM 
performs  better  against  itself,  but  Kendall’s  Tau  remains  better  with  a  higher  percentage  of  correct  estimates.  When 
the  conditions  for  correctness  are  loosened  to  the  ±2°  in  any  one  angle,  Kendall’s  Tau  is  nearly  100%  correct  for  this 
type  of  satellite  in  this  geometric  configuration,  and  can  be  seen  in  Fig.  14.  SAM  has  nearly  identical  performance 
in  the  exact  and  approximate  solution  configurations  with  the  exception  of  a  few  percentage  points.  Further 
investigation  is  required  to  determine  a  cause  for  this  behavior,  and  if  the  signature  database  is  a  source  for  the 
similarity,  or  if  there  is  something  inherent  in  the  curve  matching  algorithm. 
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Figure  13.  Cylinder  Sat-Low  Elevation,  Photometric  Figure  14.  Cylinder  Sat-Low  Elevation,  Photometric 
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Photometric  Signature  Matching  Summary 


For  Cylinder  Sat,  in  a  high  elevation  pass  (greater  than  65°),  SAM  provided  better  performance.  The  results  for  the 
Cube  Sat  match  that  of  the  Cylinder  Sat,  and  is  the  main  reason  those  results  are  omitted  in  this  report.  In  lower 
elevation  passes  (below  65°  and  greater  than  50°),  Kendall’s  Tau  provided  better  performance  for  the  Cylinder  and 
Cube  satellites.  Because  of  the  different  geometric  configurations  providing  different  results  for  each  method  a 
weighting  could  be  applied  to  the  algorithms  based  on  elevation,  and  satellite  type  if  known,  to  determine  the 
algorithm  best  suited  to  the  observing  situation.  These  results  provide  a  solid  foundation  for  the  Monte  Carlo 
simulation  testing  utilizing  the  end-to-end  environment  discussed  in  the  results  section  below. 

Generalized  Classification  Accuracy  Testing 


To  test  and  verify  the  classification  accuracy  a  test  was  developed  where  an  individual  test  is  defined  by  measuring 
the  performance  of  the  algorithm  for  a  representative  set  of  satellite  passes  over  a  30-day  period  that  a  Raven  class 
optical  sensor  would  observe.  If  the  single  forward  modeled  photometric  signature  (FMPS)  solution  was  correctly 
matched,  then  the  individual  pass  will  be  said  to  be  “Correctly  Classified”.  If  an  incorrect  FMPS  was  matched,  the 
pass  will  be  labeled  as  “Mis-classified.”  The  aggregate  Classification  Accuracy  accumulated  over  the  simulated  30- 
day  period  will  be  used  to  determine  if  the  test  passed  or  failed.  The  success  criteria  of  this  testing  is  set  at  a 
classification  accuracy  of  90%  previously  stated. 

Over  a  31  day  period,  six  generalized  satellites  were  simulated  from  1  December  2009  00:00:00  to  31  December 
2009  23:59:59.  This  resulted  in  94  passes  fitting  the  parameters  of  the  pass  finder.  The  parameters  in  this  case  were 
that  the  pass  needed  to  culminate  over  the  observing  site  at  least  45  degrees  in  elevation  and  be  solar  illuminated 
while  the  observing  site  was  dark,  or  terminator  conditions.  The  sensor  location  was  set  to  be  at  the  RME  in  Kihei, 


Maui.  To  perform  the  test  each  satellite  pass  was  given  four  signatures  to  be  compared.  These  signatures  consisted 
of:  the  satellite  observing  the  observer,  a  signature  that  was  within  two  degrees  of  the  observer,  the  nadir  signature 
and  a  randomly  chosen  signature  that  was  not  part  of  the  previous  three  photometric  signatures  based  on  the  angles 
described  earlier.  Together  these  four  signatures  provide  different  test  cases  for  the  photometric  signature  matching 
algorithm  and  the  satellite  orientation  estimation  accuracy. 

For  each  test  case  the  satellite  pass  information  was  read  into  a  software  suite  that  contains  the  algorithms.  The 
software  suite  is  responsible  for  generating  the  TASAT  signatures,  or  the  forward  modeled  photometric  signatures 
(FMPS),  and  the  Kendall’s  Tau  surface  maps,  and  feeds  directly  into  the  signature  matching  algorithms.  To 
thoroughly  test  the  robustness  of  the  algorithms,  each  individual  test  on  each  signature  used  a  random  calibration 
error  value  in  the  bounds  of  [-20,  20]  percent.  In  addition  to  the  calibration  error,  zero-mean  random  Gaussian  noise 
was  also  added  to  the  measured  photometric  signature  (MPS).  After  degradation  of  the  MPS,  curve  matching  was 
performed  for  each  of  the  four  photometric  signatures  for  each  satellite  pass,  or  simulation.  When  performed  over 
the  94  different  satellite  passes  there  were  a  total  of  376  different  photometric  signature  matches  performed.  This 
test  was  performed  50  times  for  a  total  of  18,800  realizations  of  a  MPS  to  be  compared  to  each  simulated  FMPS. 
The  results  of  these  tests  are  summarized  in  Table  1.  The  Classification  Accuracy  metric  is  shown  below  in  (3)  and 
is  displayed  in  the  last  column  of  Table  1. 

2j  ( IncorrectSolutions  +  MultipleSolutions  +  NoSolutions) 

ClassificationAccuracy  =  1 00  - -  (3) 

TotalClassifcationsAttempted 


Table  1.  Testing  Results  for  Classification  Accuracy  of  software  suite  with  curve  matching  algorithms. 


Description  of  Test 

Number  Correct 

Classification  Accuracy 

Completely  Correct  Satellite  orientation 
Estimates 

-Both  Methods  Agree 

17145 

91.2% 

Correctly  Estimated  by  TAU  Method 
-TAU  Correct,  SAM  Incorrect 

17220 

91.6% 

Correctly  Estimated  by  SAM  Method 
-SAM  Correct,  TAU  Incorrect 

18668 

99.3% 

Correctly  Estimated  by  Either  Method 
-SAM  Correct  or  TAU  Correct 

18762 

99.8% 

To  present  the  data  in  another  way,  a  bar  graph  depicting  the  percentage  of  satellite  orientations  correctly  estimated 
is  shown  in  Figure  15.  Overlaid  on  the  bar  chart  are  two  other  plots  showing  the  actual  number  of  satellite 
orientations  incorrectly  estimated,  for  each  376  different  comparisons,  and  the  number  of  incorrectly  estimated 
satellite  orientations  when  neither  algorithm  got  the  solution  correct.  Drawn  across  the  top  of  the  chart  is  the  success 
rate  of  90%.  Many  of  the  different  tests  are  above  this  threshold,  but  a  few  are  not.  Overall,  the  software  and  the 
algorithms  were  able  to  correctly  estimate  91.2%  of  all  the  satellite  orientations  when  both  the  Kendall’s  Tau  and 
Spectral  Angle  Mapper  agreed  on  the  satellite  orientation. 

Since  there  were  never  “No  Solutions”,  as  the  software  always  chooses  a  solution,  then  the  problem  reverts  to  when 
the  two  methods  disagree  on  the  estimated  satellite  orientation.  If  we  look  at  the  data  when  there  were  “Multiple 
Solutions”  chosen  by  the  software,  it  is  important  to  see  how  many  of  those  were  actually  correctly  estimated  by  one 
of  the  algorithms.  The  data  shown  in  Figure  16  is  the  number  of  correctly  estimated  satellite  orientations  by  either 
one  of  the  algorithms  and  the  number  of  incorrect  solutions  is  overlaid.  By  using  this  method,  the  percent  of 
correctly  estimated  satellite  orientations  jumps  to  99%  correct. 

Classification  Accuracy  Summary 

In  summary,  the  correct  satellite  orientation  was  estimated  correctly  greater  than  90%  of  the  time,  which  was  the 
goal  of  this  project.  A  satellite  orientation  is  correctly  estimated  when  both  algorithms  agree  on  the  satellite 
orientation.  A  satellite  orientation  is  marked  as  “Multiple  Solutions”  when  the  two  algorithms  disagree  on  the 


satellite  orientation.  When  the  algorithms  did  disagree  on  the  satellite  orientation  estimate,  99.8%  of  the  time  one  of 
the  two  algorithms  got  the  correct  solution.  This  means  that  there  was  only  0.2%,  or  36  passes,  incorrectly  classified 
out  of  18,800. 


Figure  15.  Results  for  the  Classification  Accuracy  Figure  16.  Results  for  the  Classification  Accuracy 
testing  when  both  methods  agree  on  satellite  orientation  testing  when  either  of  the  two  algorithms  correctly 
estimate.  estimates  satellite  orientation  within  two  degrees. 


5.  Conclusion 

This  feasibility  study  focused  on  the  use  of  forward  modeled  photometric  signatures  and  two  curve  matching 
techniques  to  determine  a  best  fit  solution  to  a  measured  signature  of  a  satellite.  The  biggest  assumption  made  is  that 
the  photometric  technique  used  on  the  sensor  accounts  for  the  calibration  errors  in  the  system,  leaving  the 
atmospheric  transmission  as  the  only  source  of  corruption.  This  corruption  is  introduced  as  a  erroneous  estimate  in 
the  extinction  coefficient  off  of  a  nominal  value  of  0.17.  At  less  than  +10%  atmospheric  calibration  errors  the  SAM 
method  is  able  to  correctly  identify  greater  than  95%  of  passes  for  a  cylindrical  satellite  during  a  high  elevation  pass 
as  seen  in  Figure  11.  By  lifting  the  pose  estimate  restricts  to  2°  in  one  angle,  the  ability  of  SAM  to  correctly  estimate 
poses  with  95%  accuracy  expands  to  +30%  atmospheric  calibration  errors.  If  the  satellite  is  in  a  lower  elevation 
geometry,  the  Kendall’s  Tau  method  is  then  the  best  estimator  for  poses.  This  is  shown  in  Figure  13  and  the 
Kendall’s  Tau  correctly  estimates  poses  95%  of  the  time  with  calibration  errors  up  to  +20%.  When  using  the  two 
methods  in  combination  with  each  other  over  a  Monte  Carlo  simulation  of  18,800  different  signatures,  the  post  was 
correctly  estimated  99.8%  of  the  time.  Kendall’s  Tau  and  the  Spectral  Angle  Mapper  provide  a  feasible  way  of 
comparing  measured  signatures  to  forward  modeled  photometric  signatures  with  a  high  degree  of  accuracy,  given 
that  the  forward  model  is  accurate. 
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